Method for automated integration testing with mock microservices

ABSTRACT

A method for integration-testing a microservices-based application is performed by an automated software-testing system by using mock software entities. The mock entities emulate the behavior of other microservices or external software applications that would normally interact with the tested microservices in a production environment. The mock entities are managed by a mock server that communicates with the test system through a messaging system. The mock server determines how the mock entities should respond to the tested application&#39;s service requests by referring to test guidelines stored in a behavior store. During testing, the test system dynamically updates the behavior store in response to the tested application&#39;s service requests and to other indicators of the current state of the tested application. If the tested application interacts with the mock entities during the test as expected, the test system deems the application to have passed the integration test.

This application is a continuation application claiming priority to Ser.No. 15/817,365, filed Nov. 29, 2017.

BACKGROUND

The present invention relates in general to automated testing ofsoftware applications and in particular to a method for integrationtesting of applications that are based on a microservices architecture.

A microservices architecture distributes functionality of an applicationacross a set of interconnected standalone processes calledmicroservices. Each microservice may operate as a black box, providingdistinct functionality that is exposed to other microservices by astrictly defined interface.

This allows each microservice to be tested in isolation beforeperforming testing the entire application as an integrated set ofmicroservices Isolation testing a microservice is relativelystraightforward, using dummy “mock” processes to submit input to themicroservice through its formal interface and then determining whetherthe microservice responds through the interface with expected output.

Integration testing involves comprehensive end-to-end testing of theentire application, including testing of the application's interactionswith external systems. In some cases, integration testing may comprisetesting each subsystem of the application, where each subsystem consistsof a subset of the application's microservices. Testing a subsystem inthis way requires creation of functioning “mock” microservices thatexchange data with the subsystem.

Tools exist for creating mock processes to be used in isolation testingof a single microservice. However, because testing a subsystem can be somuch more complex than testing an isolated microservice or component,there is no easy way to generate an integrated set of microservicesnecessary to perform integration testing on a subsystem or applicationthat itself comprises many microservices.

SUMMARY

An embodiment of the present invention provides a method for automatedintegration testing with mock microservices, the method comprising aspecialized test system, of an automated application-testing mechanism,performing integration testing on microservice-based applications,receiving a request from the automated testing mechanism to perform anintegration test on an integrated subsystem of the microservice-basedapplication. In response, the integration-test system launches a mockserver that creates and manages mock microservices and applications.These mock entities emulate the operation of corresponding real-worldentities with which the tested subsystem would normally interact afterdeployment. The mock entities intercept service requests from the testedsubsystem and forward them through a messaging system to the testsystem. The test system determines an expected response to each servicerequest by referring to guidelines stored in a behavior store. The testsystem may dynamically update this behavior store during the integrationtest as required in order to identify authentic responses to particularservice requests. At the conclusion of the test, the system analyzes itscommunications with the tested subsystem to determine whether theintegrated subsystem exhibited expected behavior during testing. If so,the test system notifies its parent automated application-testingmechanism that the tested subsystem has passed the integration test.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the structure of a computer system and computer programcode that may be used to implement a method for automated integrationtesting with mock microservices in accordance with embodiments of thepresent invention.

FIG. 2 shows an architecture of a microservices integration-test systemin accordance with embodiments of the present invention.

FIG. 3 shows an example of a test script in accordance with embodimentsof the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention improve existing technologies forthe automated integration testing of a software application thatcomprises a microservices architecture. As explained above, integrationtesting requires concurrently testing all, or a significant subset ofall, the microservices comprised by the application.

Known methods of integration testing comprise a mock server that hasbeen manually programmed to imitate the behavior of services orapplications extrinsic to the application being tested or to the subsetof microservices being tested (referred to here, for the purpose ofsimplicity, as “the tested application”). This mock server tests thetested application by communicating to the tested application input datathat the application would receive from the extrinsic services orapplications in a real-world implementation. The mock server, or anothercomponent of the automated test system, then verifies the correctoperation of the tested application by determining whether the testedapplication responds to the input data by generating expected output.

The mechanism has several disadvantages. It requires the mock server tobe manually programmed to produce all possible types of input that thetested application might be expected to receive. If the automated testsystem uses system calls or standard APIs to query the mock server, inorder to determine whether the tested application has respondedcorrectly, then an automated integration procedure may require a largenumber of synchronous system calls or API calls.

These known automated integration-testing mechanisms also require thatthe integration tests be written in a manner that differs considerablyfrom unit tests and component tests that might test a single microsmice.Because a unit-test or component-test system runs all code required totest a microservice without requiring other microservices to beoperational, such a test system has direct, synchronous access to boththe code being run and to any mock microservice that would normally runthat code. This type of testing is considerably different than automatedintegration testing, where a mock server runs independently of the testsystem. While an automated integration-testing requires manuallyprogramming test behavior into the mock server in advance, an automatedunit-testing application can dynamically adapt its test proceduresduring testing as a function of a tested unit's responses to the mockservices.

Other drawbacks of known automated integration-testing systems include:

-   -   the need for the mock server, not the test system, to manage the        mock server's behavior during testing. This is necessary in        order to allow multiple instances of the mock server to operate        concurrently when exhibiting identical test behavior;    -   a requirement for the test system to continuously poll the mock        server or some other monitoring component in order to keep track        of the current state of the mock server;    -   a requirement for the test system to have knowledge of the        logical or physical addresses of the tested application; and    -   burdensome complexity when attempting to concurrently run        several tests that each require the mock server to exhibit        different behavior.

The present invention comprises a software module of an automatedapplication-testing mechanism that is adapted to provide a moreefficient, accurate, and robust method for performing integrationtesting upon applications designed with a microservices architecture.This inventive concept improves upon existing microserviceintegration-testing functions of application-testing technologies bystoring testing behavior in a “behavior store” component of theintegration-testing system, rather than in the mock server. As a result,when the mock server receives a service request or other output from thetested application, it is the test system, not the mock server itself,that determines the mock server's response. This allowsintegration-testing procedures to be designed and performed in a manneranalogous to those of unit tests.

Another way that the present invention improves current automatedintegration-testing technology is by providing an integration-testingsystem architecture comprising a mock server that requests directionfrom the test system whenever a tested application requires a responsefrom the mock server. In other words, when the mock server receives aservice request from the tested application, the mock server determineshow to service that request by making a corresponding request to thetest system.

The test system determines how to service such requests from the mockserver by referring to an integrated behavior store that identifies howthe mock server should respond to each type of call or request receivedfrom the tested application. The test system may dynamically reconfigurethe behavior store as the test runs, allowing the mock server's behaviorto be modified on the fly in response to changing test conditions and inresponse to requests, functional output, messages, and other contentreceived from the mock server.

Another improvement is that the behavior store may interface to othercomponents of the test system a manner similar to that used for mockservices during unit testing. For example, the test system may query orotherwise access the behavior store by means of calls to existingprocess-mocking libraries like the JavaScript unit-test developmentfacility Sinon.JS. This allows the present invention to run integrationtest scripts by means of technologies and techniques similar to thoseused to generate and run unit tests. This advantage also facilitatesdevelopment and maintenance of integration test scripts by allowingintegration tests to be written in formats and styles similar to thoseof unit tests. For all these reasons, the present invention allows thedevelopment of integration test scripts that comprehensively describe atested unit's expected behavior under test in an easily understandablemanner.

Finally, because the mock server is stateless, the present inventionallows multiple instances of a mock server to run concurrently withoutrequiring a distinct process or service to store each instance's currentstate. This is an important advantage on cloud-computing platforms whenintegration testing requires multiple copies of a test-system service.Requiring the provisioning of an additional service to store eachmock-server state would add testing complexity by introducing additionalpoints of failure. This advantage is even more important if the mockservice is used to test the tested application in a productionenvironment.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computerdable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

FIG. 1 shows a structure of a computer system and computer program codethat may be used to implement a method for automated integration testingwith mock microservices in accordance with embodiments of the presentinvention. FIG. 1 refers to objects 101-115.

In FIG. 1, computer system 101 comprises a processor 103 coupled throughone or more I/O Interfaces 109 to one or more hardware data storagedevices 111 and one or more I/O devices 113 and 115.

Hardware data storage devices 111 may include, but are not limited to,magnetic tape drives, fixed or removable hard disks, optical discs,storage-equipped mobile devices, and solid-state random-access orread-only storage devices. I/O devices may comprise, but are not limitedto: input devices 113, such as keyboards, scanners, handheldtelecommunications devices, touch-sensitive displays, tablets, biometricreaders, joysticks, trackballs, or computer mice; and output devices115, which may comprise, but are not limited to printers, plotters,tablets, mobile telephones, displays, or sound-producing devices. Datastorage devices 111, input devices 113, and output devices 115 may belocated either locally or at remote sites from which they are connectedto I/O Interface 109 through a network interface.

Processor 103 may also be connected to one or more memory devices 105,which may include, but are not limited to, Dynamic RAM (DRAM), StaticRAM (SRAM), Programmable Read-Only Memory (PROM), Field-ProgrammableGate Arrays (FPGA), Secure Digital memory cards, SIM cards, or othertypes of memory devices.

At least one memory device 105 contains stored computer program code107, which is a computer program that comprises computer-executableinstructions. The stored computer program code includes a program thatimplements a method for automated integration testing with mockmicroservices in accordance with embodiments of the present invention,and may implement other embodiments described in this specification,including the methods illustrated in FIGS. 1-3. The data storage devices111 may store the computer program code 107. Computer program code 107stored in the storage devices 111 is configured to be executed byprocessor 103 via the memory devices 105. Processor 103 executes thestored computer program code 107.

In some embodiments, rather than being stored and accessed from a harddrive, optical disc or other writeable, rewriteable, or removablehardware data-storage device 111, stored computer program code 107 maybe stored on a static, nonremovable, read-only storage medium such as aRead-Only Memory (ROM) device 105, or may be accessed by processor 10′3directly from such a static, nonremovable, read-only medium 105.Similarly, in some embodiments, stored computer program code 107 may bestored as computer-readable firmware 105, or may be accessed byprocessor 103 directly from such firmware 105, rather than from a moredynamic or removable hardware data-storage device 111, such as a harddrive or optical disc.

Thus the present invention discloses a process for supporting computerinfrastructure, integrating, hosting, maintaining, and deployingcomputer-readable code into the computer system 101, wherein the code incombination with the computer system 101 is capable of performing amethod for automated integration testing with mock microservices.

Any of the components of the present invention could be created,integrated, hosted, maintained, deployed, managed, serviced, supported,etc. by a service provider who offers to facilitate a method forautomated integration testing with mock microservices. Thus the presentinvention discloses a process for deploying or integrating computinginfrastructure, comprising integrating computer-readable code into thecomputer system 101, wherein the code in combination with the computersystem 101 is capable of performing a method for automated integrationtesting with mock microservices.

One or more data storage units 111 (or one or more additional memorydevices not shown in FIG. 1) may be used as a computer-readable hardwarestorage device having a computer-readable program embodied thereinand/or having other data stored therein, wherein the computer-readableprogram comprises stored computer program code 107. Generally, acomputer program product (or, alternatively, an article of manufacture)of computer system 101 may comprise the computer-readable hardwarestorage device.

While it is understood that program code 107 for a method for automatedintegration testing with mock microservices may be deployed by manuallyloading the program code 107 directly into client, server, and proxycomputers (not shown) by loading the program code 107 into acomputer-readable storage medium (e.g., computer data storage device111), program code 107 may also be automatically or semi-automaticallydeployed into computer system 101 by sending program code 107 to acentral server (e.g., computer system 101) or to a group of centralservers. Program code 107 may then be downloaded into client computers(not shown) that will execute program code 107.

Alternatively, program code 107 may be sent directly to the clientcomputer via e-mail. Program code 107 may then either be detached to adirectory on the client computer or loaded into a directory on theclient computer by an e-mail option that selects a program that detachesprogram code 107 into the directory.

Another alternative is to send program code 107 directly to a directoryon the client computer hard drive. If proxy servers are configured, theprocess selects the proxy server code, determines on which computers toplace the proxy servers' code, transmits the proxy server code, and theninstalls the proxy server code on the proxy computer. Program code 107is then transmitted to the proxy server and stored on the proxy server.

In one embodiment, program code 107 for a method for automatedintegration testing with mock microservices is integrated into a client,server and network environment by providing for program code 107 tocoexist with software applications (not shown), operating systems (notshown) and network operating systems software (not shown) and theninstalling program code 107 on the clients and servers in theenvironment where program code 107 will function.

The first step of the aforementioned integration of code included inprogram code 107 is to identify any software on the clients and servers,including the network operating system (not shown), where program code107 will be deployed that are required by program code 107 or that workin conjunction with program code 107. This identified software includesthe network operating system, where the network operating systemcomprises software that enhances a basic operating system by addingnetworking features. Next, the software applications and version numbersare identified and compared to a list of software applications andcorrect version numbers that have been tested to work with program code107. A software application that is missing or that does not match acorrect version number is upgraded to the correct version.

A program instruction that passes parameters from program code 107 to asoftware application is checked to ensure that the instruction'sparameter list matches a parameter list required by the program code107. Conversely, a parameter passed by the software application toprogram code 107 is checked to ensure that the parameter matches aparameter required by program code 107. The client and server operatingsystems, including the network operating systems, are identified andcompared to a list of operating systems, version numbers, and networksoftware programs that have been tested to work with program code 107.An operating system, version number, or network software program thatdoes not match an entry of the list of tested operating systems andversion numbers is upgraded to the listed level on the client computersand upgraded to the listed level on the server computers.

After ensuring that the software, where program code 107 is to bedeployed, is at a correct version level that has been tested to workwith program code 107, the integration is completed by installingprogram code 107 on the clients and servers.

Embodiments of the present invention may be implemented as a methodperformed by a processor of a computer system, as a computer programproduct, as a computer system, or as a processor-performed process orservice for supporting computer infrastructure.

FIG. 2 shows an architecture of a microservices integration-test systemin accordance with embodiments of the present invention. FIG. 2 showselements identified by reference numbers 200-260 and 2000-2003.

The exemplary system of FIG. 2 may perform integration testing of amicroservices-based tested application 2001. This testing may be part ofa development or implementation workflow, in order to verify the correctoperation of a new application, or during actual production, to confirmthe current health of an installed system.

FIG. 2 shows a test system 2000 that performs integration testing on amicroservices-based tested application 2001. During testing, a mockserver 2002 directs “mock” software entities 230-240 to exchange servicerequests 250 and responses 260 with tested application 2001. Thisprocedure is controlled by test system 2000, which communicates withmock server 2002 through a messaging system 2003. Test system 2000identifies responses 260 that allow each mock entity 230-240 to emulatebehavior that the tested application 2001 would receive fromcorresponding real-world systems during actual production operation.

Test system 2000 comprises a test script 200 that describes stepscomprised by the integration test, and a behavior store 210 thatidentifies how the mock software entities 230-240 should respond toservice requests 250 received from tested application 2001 duringtesting. Test system 2000 uses the test script 200 to continually updatethe behavior store 210 in response to receiving service requests 250 andin response to changes in the state of the tested application 2001.

The tested application 2001 is composed of a set of microservices thateach performs a discrete set of operations. Each microservice mayfunction independently of other microservices, interoperating as blackboxes that communicate only through formal interfaces.

Integration testing verifies the correction operation of a subset 220 ofthese microservices by testing the tested subset 220 in aggregate as asingle integrated software object. This procedure comprises receivingand responding to service requests sent by the subset 220 and verifyingthat the tested subset 220 behaves correctly under real-worldconditions. In some cases, a subset may comprise the entire testedapplication 2001.

Any microservices of the tested application 2001 that are not beingtested may exchange formal communications with mock microservices 230that emulate the operation of the untested microservices. These mockmicroservices 230 communicate with the tested microservices 220 duringtesting through the tested application 2001's standard internalinterfaces.

External applications, computerized systems, physical or virtualinfrastructure components, or other extrinsic entities with which testedapplication 2001 might be expected to communicate during normaloperation may likewise be replaced by mock applications 240 that emulatethe operation of such extrinsic entities. Like mock microservices 230,mock applications 240 communicate with the tested application 2001through the standard interfaces originally designed into testedapplication 2001.

Each mock microservice 230 and mock application 240 is thus programmedto act as a proxy for a real-world internal microservice or externalentity, exchanging communications with the tested microservices 220through those components' standard interfaces, so as to duplicate theactual operating environment of the tested components. By observing theoperation of the tested microservices 220 under these conditions, testsystem 2000 may then determine whether the tested microservices 220interact correctly with their environment under normal operatingconditions.

Mock microservices 230 and mock applications 240 are created and managedby mock server 2002 in accordance with rules, guidelines, or otherdescriptions of the mocked components' behavior contained in behaviorstore 210. This decoupled architecture allows test system 2000 to updatebehavior store 210 on the fly during testing in response to a particularrequest 250 made by tested application 2001. Embodiments of the presentinvention can thus react to the behavior of tested application 2001 bydynamically changing the behavior of test system 2000 during the actualperformance of an integration test.

Embodiments of the present invention are flexible enough to accommodateany communications mechanism between test system 2000 and mock server2002. In the example of FIG. 2, this communications mechanism comprisesa discrete messaging system 2003 (such as a pub/sub system or pollingsystem) that exchanges communications between behavior store 210 (whichidentifies how mock components 230 and 240 should respond to servicerequests 250 received from tested application 2001), and mock server2002 (which directly controls the operation of mock microservices 230and mock applications 240).

This decoupled approach offers the following advantages:

-   -   neither test system 2000 nor mock server 2002 needs to be        programmed with each other's addresses;    -   communications between test system 2000 and mock server 2002 are        event-driven, thus allowing test program 2000 to always be        available to service tested application 2001; and    -   the integration-testing platform as a whole is more tolerant of        temporary network outages.

In FIG. 2, when the tested application 2001 communicates a servicerequest 250 to a mock microservice 230 or mock application 240, the mockserver responds by forwarding the service request 250 to test system2000 through the messaging system 2003. Conversely, test system 2000responds to a service request 250 by returning a service response 260 tothe mock server 2002 through the messaging system 2003. Mock server 2002then directs a mock microservice 230 or mock application 240 to returnthe response to the tested application 2001. In both cases, the servicerequest 250 or service response 260 is communicated between the mockserver 2002 and the behavior store 210 of the test system 2000 by meansof messaging system 2003.

This approach allows test system 2000 to continuously monitor the stateof tested application 2001 by tracking each service request 250 sent bya tested microservice 220 to a mock microservice 220. Test system 2000can therefore keep track of the current state of mock server 2002 and oftested application 2001 at all times during the integration-testingeffort and can use information contained in test script 200 to updatebehavior store 210 (and, indirectly, the behavior of mock microservices230 and mock applications 240) as required during testing. Thisasynchronous mechanism also allows behavior store 210 to pause testingwhile waiting for the test system 2000 to identify, from test script200, an appropriate service response 260.

FIG. 3 is a flow chart that illustrates steps of a method for automatedintegration testing with mock microservices in accordance withembodiments of the present invention. FIG. 3 contains steps 300-390.

In step 300, an integration-testing system 2000, which may beimplemented as a software module or component of a comprehensiveautomated application-testing mechanism, receives from the automatedapplication-testing mechanism a request to perform an integration teston a tested application 2001 that comprises a microservicesarchitecture. This request may include test details that may not alreadybe available to test system 2000, such as a test script 200, a testschedule, descriptions of resources to be used for testing,characteristics of formal interfaces through with the tested application2001 may communicate with external hardware or software systems, andother characteristics of the application to be tested 2001.

If the integration test will be testing only a subset 220 of themicroservices comprised by tested application 2001, the request may alsoidentify the tested subset 220, the untested microservices of testedapplication 2001, and characteristics of the interfaces through whichthe tested subset 220 communicates with the untested microservices.

In step 310, the test system 2000 sets up the integration test. Thisstep may include configuring and launching a mock server 2002 that inturn launches any mock microservices 230 and mock applications 240 thatwill communicate with the tested subset 220 of microservices.

Set-up may also comprise other configuration steps, such as initializingbehavior store 210 with test steps derived from the test script 200.This initial configuration of behavior store 210 may allow the behaviorstore 210 to respond to initial service requests 250 received from thetested subset 220.

Test system 2000 then launches the n accordance with guidelinesspecified in the received test request or in test script 200.

Step 320 begins an iterative procedure of steps 320-360. Each iterationis triggered when the test system 2000 receives a service request 250from the tested subset 2001. The iterative procedure ends at theconclusion of the integration test.

In step 330, the test system 2000 receives a service request 250 fromthe tested subset 220. This service request 250 is sent by the testedsubset 220 to a mocked entity 230 or 240 in accordance with conventionsof a standard interface defined by the design of the tested application2001.

In one example, a tested subset 220 of an e-commerce program maycomprise a subset of microservices that provide order-processingfunctionality to the e-commerce program. This tested order-processingsubsystem may be designed to communicate with a user-input microserviceof the e-commerce program and with an external database application,neither of which are subjects of the integration test. In accordancewith principles of microservice software design, the user-inputmicroservice would communicate with the tested order-processingsubsystem only through one or more strictly defined interfaces,exchanging communications as black-box software modules. All of thisinformation would be provided to the test system 2000, possibly as partof the test request received in step 300.

In step 310 of this example, the mock server 2002 would launch a mockmicroservice 230 that emulates the operation of the real-worlduser-input microservice. This mock microservice 230 would communicatewith the tested order-processing subsystem through the testedapplication's standard interface mechanism. The mock server 2002 wouldalso create and operate a mock application 240 that emulates theoperation of the real-world extrinsic database application. Again, thismock application 240 would communicate with tested subset through thetested application's standard interface. If configured correctly, thetested subset 220 would not be able to tell that the mock entities 230and 240 were not actually the real-world software modules that the mockentities replace.

In this example, step 330 might comprise the sub-steps:

-   -   a tested microservice 220 of the tested order-processing        subsystem attempts to retrieve a user account number from the        external database by sending a service request 250 that consists        of a database query. This query conforms in format and content        to a standard interface format, designed into the e-commerce        application, by which the requesting microservice normally        accesses the database;    -   the query 250 is intercepted by the mock database application        240, which operates under the control of mock server 2002    -   the mock server forwards the service request 250 to the test        system 2000. This communication may be performed through a        discrete messaging system 2003, such as a publish-subscribe        system or other type of messaging system known in the art. The        present invention is flexible enough to also accommodate any        other known communications mechanism desired by an implementer;        and    -   test system 2000 receives the service request 250 by means of        the messaging system or other communications mechanism 2003.

In optional step 340, the test system 2000 identifies the current stateof tested application 2001 or tested subset 220 of tested application2001. This identification may be performed by analyzing priorcommunications with the tested subset 220 received during theintegration test or through any other means known in the art, such as byintercepting error messages or log entries before or during theintegration test.

In step 350, test system 2000, if necessary, updates the contents ofbehavior store 210. This updating may be necessary if the behavior store210 does not contain rules, guidelines, or other content capable ofidentifying an appropriate service response 260 to service request 250.

Test system 2000 might require such updating if necessary for the testsystem 2000 to determine how to respond to tested subset 220 when testedsubset 220 is in an unexpected state. For example, if tested subset 220was determined in step 340 to be in an error state or to have detectedsuspicious user input, behavior store 210 may require additional contentthat is directed to error-management communications or to high-securityfraud-prevention protocols.

Updating may also be necessary if behavior store 210 does not containinformation pertinent to the specific service request 250 received instep 330. This may occur, for example, it a tested subsystem cangenerate so many distinct service requests 250 that the behavior store210 does not have the capacity to store logic related to servicing everypossible request 250.

In some embodiments, this updating may be performed by transferringinformation from the test script 200 to the behavior store 210. Thisinformation may, by any means known in the art, be reformatted,abridged, elaborated, or otherwise transformed if necessary to addressspecific requirements of the most recently received service request 250.

In other cases, the updating may be performed as a function ofinformation retrieved from other sources. For example, servicing thequery service request 250 for a user account number might require thetest system 2000 to query a real database application in order toretrieve an actual account number or a dummy number that is formatted inan expected manner.

Other embodiments may use any means known in the art to dynamicallyupdate behavior store 210 during integration testing, as needed in orderto ensure that behavior store 210 responds to a service request 250 withan appropriate and authentic service response 260.

In step 360, test system 2000 responds to the service request 250received in step 340 by returning an expected service response 260 tothe tested subset of microservices 220. This response is selected by thetest system 2000, as a function of the current contents of the behaviorstore 210, so as to most closely match a response that would be expectedfrom the mock entity 230 or 240 to which the service request 250 hadoriginally been directed.

In the preceding example, this response 260 might consist of the useraccount number requested by the tested subset 220 in step 330. Thisresponse would be returned to the requesting subset 220 through a pathsimilar to that through which the service request 250 had traveled instep 330. That is, test system 2000 would send the response 260 throughmessaging system 2003 to mock server 2002. Mock server 2002 would thendirect mock database application 240 to return the user account numberto the requesting microservice or subsystem, such that the returnedmessage conformed to conventions of the e-commerce system's standardinterface.

The iterative procedure of steps 320-360 ends when the integration testis complete. This may occur when all tasks of the test script have beenperformed or when it is determined that the test cannot proceed becauseof unexpected test conditions or test errors. At this point, the methodof FIG. 3 concludes with steps 370-390.

In step 370, test system 2000 determines whether the tested subset 220exhibited expected behavior during the integration test. This expectedbehavior may be defined by the test script 200, by a designspecification, by a set of standards or conventions, by a lack of errordetections, or by any other means known in the art. The exhibitedbehavior may comprise the set or sequence of service requests 250 orother communications received from the tested subset 220, or maycomprise other received or retrieved information about the state of thetested subset 220 during the integration test.

In step 380, if the test system 2000 in step 370 determined that thesubset 220 correctly exhibited expected behavior during the integrationtest, test system 2000 notifies the automated application-testingmechanism that the tested subset 220 has passed the integration test.

In step 390, if the test system 2000 in step 370 determined that thesubset 220 did not correctly exhibit expected behavior during theintegration test, test system 2000 notifies the automatedapplication-testing mechanism that the tested subset 220 has failed theintegration test.

Examples and embodiments of the present invention described in thisdocument have been presented for illustrative purposes. They should notbe construed to be exhaustive nor to limit embodiments of the presentinvention to the examples and embodiments described here. Many othermodifications and variations of the present invention that do not departfrom the scope and spirit of these examples and embodiments will beapparent to those possessed of ordinary skill in the art. Theterminology used in this document was chosen to best explain theprinciples underlying these examples and embodiments, in order toillustrate practical applications and technical improvements of thepresent invention over known technologies and products, and to enablereaders of ordinary skill in the art to better understand the examplesand embodiments disclosed here.

What is claimed is:
 1. A method for automated integration testing withmock microservices, the method comprising: accepting from an automatedapplication-testing system, by a processor, a request to perform anintegration test upon a tested set of microservices, where the testedset of microservices is comprised by a tested application; launching, bythe processor, a mock server; directing, by the processor, the mockserver to launch and maintain a mock entity, where the mock entityemulates an operation of a software entity that is not a subject of theintegration test, and where the software entity is known to interactwith at least one microservice of the set of microservices in aproduction environment; receiving, by the processor, from the mockserver a first service request that had been communicated to the mockentity by the tested set; sending, to the tested set, by the processor,in response to the receiving, a service response to a most recentlyreceived service request, where the sending comprises communicating theservice response to the mock server and the mock server directing themock entity to forward the service response to the tested set;receiving, by the processor, an additional service request, where theadditional service request is received at a time later than a time ofthe receipt of the first service request; repeating, by the processor,the sending in response to the processor receiving the additionalservice request; determining, by the processor, from the receiving ofthe additional service request, that the test set is operatingcorrectly; and returning, by the processor, to the automatedapplication-testing system a notification that the tested set has passedthe integration test.
 2. The method of claim 1, where the tested setcomprises all microservices of the tested application, where eachmicroservice of the all microservices functions independently of anyother microservice of the all microservices, and where the mock entityemulates a software entity that is capable of communicating with thetested application.
 3. The method of claim 1, where the mock entityemulates an untested microservice of the tested application, where theuntested microservice is a microservice that is not comprised by thetested set, and where the untested microservice is capable ofcommunicating with a tested microservice of the tested set.
 4. Themethod of claim 1, where the processor communicates with the mock serverthrough a messaging system.
 5. The method of claim 4, where themessaging system is a publish-subscribe system.
 6. The method of claim1, where the automated application-testing system comprises a behaviorstore, and where each service response sent by the processor to the mockserver is selected as a function of contents of the behavior store. 7.The method of claim 6, where the automated application-testing systemfurther comprises a test script, and where the processor dynamicallyupdates the contents of the behavior store during the integration testas a function of the test script.
 8. The method of claim 7, where thedynamic updating is performed as a further function of a current stateof the tested set.
 9. The method of claim 7, where the dynamic updatingis performed in response to communications received from the tested set.10. The method of claim 1, further comprising providing at least onesupport service for at least one of creating, integrating, hosting,maintaining, and deploying computer-readable program code in thecomputer system, wherein the computer-readable program code incombination with the computer system is configured to implement theaccepting, the launching, the directing, the receiving, the sending, therepeating, the determining, and the returning.